1. Field of the Invention
The present invention relates to image processing for detecting an image region that exhibits some poor color tone of an eye image.
2. Description of the Related Art
A method of correcting the poor color tone of an eye image due to a photographic flash of a camera has been proposed. Note that the poor color tone of an eye image is generally well known as a red-eye effect. The red-eye effect is as follows. That is, upon photographing a person or an animal such as a dog, cat, or the like using a photographic flash under an insufficient illumination environment, the flash light that has entered the opening pupil part is reflected by the eyeground, and the capillaries gleam red. Since a person who has a pale pigment color (or a light pigment) has a higher transmittance of the pupil, i.e., crystal lens, the red-eye effect tends to occur more frequently with such person.
In recent years, digital cameras are increasingly downsized, and the optical axis of a lens tends to be near the light source position of a photographic flash. In general, as the light source position of the photographic flash is closer to the optical axis of the lens, the red-eye effect occurs more readily, and a measure against this phenomenon is a critical issue.
As one means for preventing the red-eye effect, a method of performing pre-light emission before photographing, and taking a picture after the pupil of the object is closed is known. However, this method consumes a battery more than normal photographing, and changes the expression of the object due to pre-light emission.
Many methods of compensating for a red-eye image by correcting and modifying digital image data photographed by a digital camera using a personal computer have been proposed.
The methods of correcting the red-eye effect on digital image data are roughly classified into manual correction, semiautomatic correction, and automatic correction.
With the manual correction method, the user designates and corrects a red-eye region displayed on a display using a pointing device such as a mouse, stylus, tablet, or the like, or a touch panel.
With the semiautomatic correction method, the user roughly designates a region where a red-eye image exists, and correction is applied by specifying a correction range of the red-eye image based on that information. For example, the user designates a region that surrounds the two eyes using a pointing device, or designates one point near the eye. Based on information of this designated region or designated point, the correction range is specified to apply correction.
With the automatic correction method, a correction region is automatically detected from digital image data and correction processing is executed without any specific operation by the user.
Upon examining these correction methods, the automatic correction method which does not require any troublesome operations for the user and can be executed using even a device which does not comprise any high-performance display with high visibility seems most useful. However, it is not guaranteed for the automatic correction method to detect a red-eye region from an image without fault, and a red region other than a red-eye image may be erroneously detected and corrected. For this reason, image edit software or the like mounts a function of correcting a red-eye image by combining automatic correction and manual or semi semiautomatic correction. The user applies automatic correction to an image from which a red-eye image is to be corrected, and performs manual correction if he or she is dissatisfied with that result. In other words, the manual correction is an important function that compensates for the automatic correction precision.
The following techniques have been disclosed about the manual correction and semiautomatic correction methods.
A technique disclosed by U.S. Pat. No. 5,130,789 makes the user designate a region including a red-eye image to be corrected, and also designate one point in the red-eye image. The technique sets the color of a pixel of this point as a target color, and generates a geometric shape to have the target color as the center. The technique then checks for each pixel in the region designated by the user if the shape includes the color of an interest pixel, and determines pixels whose colors are included in the shape as those which form the red-eye image (to be referred to as “red-eye pixels” hereinafter), thus correcting these pixels.
Also, a technique disclosed by Japanese Patent Application Laid-Open No. 6-350914 makes the user designate an outline of a correction region or one point near the correction region using a light pen or the like. If the values of color components (e.g., RGB components) of an interest pixel included in the correction region fall within a predetermined range, the technique determines that the interest pixel is a candidate pixel which forms the red-eye image, and then checks if a set (region) of candidate pixels is a red-eye image. Then, the technique corrects the set (region) of candidate pixels determined as the red-eye image to black.
When the user designates a region including an eye, a technique disclosed by Japanese Patent Application Laid-Open No. 10-75374 checks based on the hue and saturation values of the designated region if a red-eye image exists in that region. If it is determined that a red-eye image in fact exists, the technique corrects that region. Furthermore, the technique detects catch light of an eye region, i.e., a light spot where flash light is reflected, and emphasizes the catch light if it is determined that the catch light is weak.
A technique disclosed by Japanese Patent Application Laid-Open No. 2003-304555 makes the user designate an image region including an eye, and also at least one point in the red-eye image. The technique compares the red levels of neighboring pixels in the region designated by the user, and extracts one or a plurality of red boundaries. The technique checks if the designated point falls within the red boundary. If the designated point does not fall within the red boundary, the technique does not determine the interior of the boundary as a red-eye image. That is, this method allows manual correction with high certainty.
According to the research conducted by the present applicant, although a red-eye image is a local region, it has a broad luminance distribution from a low luminance level to a high luminance level, and its tint also ranges from high-saturation red to low-saturation red. That is, the red-eye image has the property of abrupt changes in luminance and tint although it appears locally. The aforementioned technique determines based on the pixel of one point designated by the user a criterion to determine whether or not the interest pixel is a red-eye pixel. In other words, the above technique extracts pixels having a luminance and tint similar to those of one point designated by the user as red-eye pixels using that point as a criterion. However, it is difficult to extract the pixels of the red-eye image having the above property using one point as a criterion.
The above technique uses the distance between the tint of an interest pixel and a target color. However, based on such simple criterion, it is difficult to reliably extract the pixels of the red-eye image with the above property.
Furthermore, the above technique uses hue and saturation to determine whether or not the interest pixel is a red-eye pixel. However, this method suffers the following problems.
As is well known, when a pixel value is given by an RGB system, a saturation value S is given by:S={max(R,G,B)−min(R,G,B)}/max(R,G,B)  (1)where max(R, G, B) is the maximum value of RGB components, and
min(R, G, B) is the minimum value of RGB components.
For example, as is apparent from the experimental results, the flesh color regions of Japanese people are normally distributed around 0 to 30° in the hue domain (0 to 359°). Note that a hue angle of an HSI system near 0° is red, and it becomes more yellowish with increasing hue angle. The magnitude relationship of RGB values around 0 to 30° satisfies:R>G>B  (2)
As described above, in case of a person who has a dark pigment color (or a rich pigment), a bright red-eye image hardly tends to form compared to a person who has a pale pigment color (or a light pigment).
In consideration of these facts, the red-eye pixel values and the pixel values of the flesh color region around the eyes of the Japanese can be estimated as:
Red-eye region: (R, G, B)=(109, 58, 65)
Flesh color region: (R, G, B) (226, 183, 128)
In the above case, the saturation value of red-eye pixels is 40, and that of pixels of the flesh color region is 43, i.e., these saturation values are roughly equal to each other. In other words, if pixels which have saturation values within a given range having the saturation value of the target color as the center are determined as red-eye pixels, the pixels of the flesh color region are also more likely to be determined as red-eye pixels.